Method and apparatus for progressively deleting media objects from storage

ABSTRACT

A system for managing storage space on an electronic storage medium is provided in which a file format for stored data allows for progressive deletion of low-significance data, for example in a video or audio file, while allowing the remaining portions of the file to be subsequently retrieved. The file format allows for the ready deletion of low-significance data without having to open, edit and subsequently rewrite the data. Furthermore, rules-based algorithms for the deletion of low-significance data allow a user to store and progressively delete such low-significance data in accordance with time parameters, available storage space and the like, without having to delete the full file.

This application claims the benefit of U.S. Provisional Application No.60/343,804, filed Dec. 27, 2001, which is incorporated herein byreference.

FIELD OF THE INVENTION

The present invention is directed generally to the storage of mediaobjects such as digital video or audio files or streams, and moreparticularly to the use of variable levels of compression for flexibleand efficient storage of collections of such objects, such as inentertainment systems and audio/video libraries.

BACKGROUND OF THE INVENTION

Effective and flexible storage of collections of audio and video contentobjects has always been a challenge because of the large number of suchobjects retained, even by an individual or family. The migration of suchcontinuous media audio or video content to digitally coded forms, andthe related convergence of devices for storage and use of such contenthas stimulated the development of a wide range of storage systems anddevices. Various devices have been employed using both fixed media, suchas computer-style electronic storage and hard disks, and removablemedia, such as video cassette recordings (VCR), compact disc (CD),Digital Versatile Disk (DVD), removable electronic storage (such asflash memory or EEPROM, Electrically Erasable Programmable Read-OnlyMemory), magnetic cards, floppy disks, and the like.

Historically, different forms of audio and video have been handled bydifferent systems and devices, but “digital convergence” is leadingtoward common and interconnected systems for reception, storage, andplayback of all kinds of media. For example, current Digital VideoRecorders (DVRs) such as TIVO, REPLAY TV, and ULTIMATETV use PC-stylehard disks, and software exists to provide DVR functions on a personalcomputer (PC), and some such devices are packaged with TV set-top boxes,such as satellite or cable decoders.

The large amount of data bits needed to store audio and video withsatisfactory quality of reproduction presents a particular challenge inboth the storage and transmission of digital format media objects. Thishas been particularly critical in transmission, due to rigid bandwidthlimits and cost constraints that apply to radio, television, onlineservices, wireless, and the like. In response to this, the use of datacompression techniques has become an essential tool. Similar issuesapply to other media objects, such as still images.

A variety of methods are used in an effort to gain maximum reduction ofobject size with minimum loss of quality. Compression has long been usedin the coding of audio into digital form for digital telephony as wellas in modems and other forms online transmission, in the coding of imagefiles and video for faxing and online distribution, and more recentlyfor digital radio and television. Compression has also been used in dataprocessing systems, with the early impetus again coming fromtransmission constraints in early remote processing systems. The storagebenefits of compression have generally followed as a secondary benefitfrom work relating to transmission needs.

A key difference between compression in data processing systems and inmedia systems is in the applicability of lossy compression techniquesthat can reduce object size by discarding some information that causeslittle or no perceptible loss in quality. Lossless compression isusually required in data processing environments, because every bit ofdata can be highly significant. Lossless compression techniques, such asPKZip and Huffman entropy codes, work by transforming inefficient codingschemes to more efficient schemes, such as by exploiting repeatingpatterns, without loss of any bits in the inversely transformed,decompressed result.

Lossy schemes are used mainly for audio, image and video data, and aretypified by standards such as JPEG (the Joint Photographic ExpertsGroup) image format and MPEG (the Motion Picture Experts Group) videoformat, including the MP3 audio coding defined within MPEG. These lossyschemes generally work by exploiting knowledge of human perception ofsound and sight, so that some selected detail is discarded incompression and permanently lost in the decompressed result, but theperceptible impact of that loss is minimized. Such methods involve atrade-off in the amount of reduction obtained and the level ofperceptible loss of quality. The level of these tradeoffs can becontrolled by the designer of the compression scheme, or left as aparameter to be controlled at the time compression is done.

A further technique of progressive or scalable, layered compression hasbeen applied to exploit these tradeoffs, particularly as they relate totransmission and rendering of digital audio and video. JPEG standardsinclude an optional progressive format, in which the image file isstructured to contain a layered sequences of scans, so that the firstscan contains a low quality rough version of the image, followed bysuccessive scans that add more detail. This enables an onlinetransmission in which the start of the file can be received and used topresent the low quality image, while successive scans are still beingreceived and then progressively rendered to enhance quality. The viewerbenefits by not having to wait to see a rough form of the image, gainingboth real utility, in having some information, and psychological benefitin not waiting with a blank screen while all of the image data isreceived and decoded.

As transmission speeds have increased and video transmission has becomea greater challenge than transmission of stills, a variation on thistechnique that addresses motion compensation has been applied in MPEG,with some attention in MPEG-2 and greater focus in MPEG-4. Motion videocompression applies to both the spatial dimension of images and the timedimension of the series of images. The multiple images of video data areorganized into groups of frames (GOFs) and the layering of gross andrefined data is done for each GOF, addressing both spatial frame imagedata and temporal inter-frame motion differences. Layering here enablesachievement of a high degree of flexibility, in that: 1) if bandwidth islimited, even momentarily, the low-significance layers can be discardedby the transmitter or ignored by the receiver in a highly adaptivefashion, and 2) if the receiver has limited processing or workingstorage resources for decoding (or a low resolution/low fidelity outputdevice that will not benefit from high quality input), it can limititself to the high-significance layers, all while minimizing theperceived quality loss. Thus such layered coding supports flexiblescalability of a single coded form of the data to use for varyingbandwidth and decoder or viewer requirements. Fine Granular Scalability(FGS) methods are intended to allow for very flexible scalings of datatransmission to the dynamic availability of transmission bandwidth.

A further method of exploiting this scalability in video transmission,as proposed by Radha and by McCanne and others (including much of aSpecial Issue on Streaming Video in the March 2001 IEEE Transactions onCircuits and Systems for Video Technology), is that of Receiver-drivenLayered Multicast (RLM). This involves transmitting each of multiplesuch layers of video in separate multicast streams, so that differentreceivers can subscribe to only the number of streams (layers) that theyrequire to obtain the quality level they desire. This provides a highquality transmission to receivers that will benefit from it, whileaccommodating those that will not. Such methods teach that a layereddata stream be partitioned at the time of transmission to createseparate layered streams for simultaneous multicast, and that thesestreams then be reassembled on receipt for combined presentation.

As noted, the emphasis on media compression has been for transmission,rather than for storage management. Data processing systems have, incontrast, seen some attention to compression-oriented storagemanagement, but efforts to apply such methods broadly have beendisappointing, and have fallen into disuse. For example, PC storagecompression systems like STACKER, DRIVESPACE, and DOUBLESPACE wereintroduced some time ago to apply compression across various files typescommonly stored on a PC disk. Such tools were based on general-purposecompression algorithms to be used across all files on a disk, and theyused lossless compression because that was required for data processingfiles. Such methods are trouble prone, and do not produce good resultson audio and video files which are increasingly the dominating use ofstorage space. The result has been that file-type and format-specificcompression techniques have been recognized as saving storage, butattempts to apply compression broadly to storage management have notbeen seen as promising areas for development.

Thus, specific media compression methods have enabled correspondingbenefits in storage systems, even though that has not been a primaryobjective in their development. For example storage systems based onJPEG compression in digital cameras, and MPEG compression in DigitalVideo Recorders (DVRs) commonly give the user a choice ofquality/compression levels. These allow the user to determine eitherglobally or on an object-by-object basis whether they should be storedat high quality levels that consume large amounts of storage, or in morehighly compressed but lower quality form.

Unfortunately, in current systems, these quality/size trade-offs must bemade at the time that an object is stored. This is a problem, becausethe value of the object and the availability of storage space may changeover time. As a collection grows and the older items in it age andpossibly fall into disuse, their importance may diminish. At the sametime, the total amount of storage used will grow, and constraints maymake it desirable to reclaim space. In current systems, no provision ismade to change the level of compression of an object. The only way toreclaim space is to reduce quality to zero by deleting the object in itsentirety. There is, therefore, a need for methods to provide for lessdraconian, more progressive and gradual ways to adjust the spaceallocated for stored media objects in a memory device.

A simple way to do this is to take the existing objects, decompressthem, and recompress them into a more compressed form, but this takesconsiderable processing resources, and depending on the specific formatsinvolved, may produce unwanted losses of quality without compensatingreductions in size, which can result from non-reversible transformationsin the decompression-recompression process. Chandra et. al. at DukeUniversity disclose work that addresses such transcoding of images, withapplications to Web browsing for transmission, and to digital camerasfor storage. Chandra recognizes the value of being able to reduce thesize of images stored in a digital camera, so that additional picturescan be stored in cases when additional storage is not available. Thetechnique applied is to partially decompress the image, and thenre-compress from that intermediate form to a lower quality level. Thisinvolves the steps of entropy decoding, dequantization, requantization,and entropy recoding. Successive reductions may be done with successivecycles of such transcodings. The problem is that this method requiressignificant processing resources and time for the transcoding, and forthe reading and rewriting of the stored object, each time such areduction is made.

Layered, scalable compression techniques have the potential tofacilitate such an objective to the extent that a storage system canreduce the number of layers that are stored for an object, withouthaving to decompress and recompress the objects. Castor et. al. in U.S.Pat. No. 6,246,797 discloses work that addresses image file sizereduction in still and video cameras. With regard to still images,Castor observes that “[a] feature of the present invention is that theimage quality level of an image file can be lowered without having toreconstruct the image and then re-encode it, which would be costly interms of the computational resources used. Rather, the data structureswithin the image file are pre-arranged so that the image data in thefile does not need to be read, analyzed or reconstructed The imagequality level of an image file is lowered simply by keeping an easilydetermined subset of the data in the image file and deleting theremainder of the data in the image file, or equivalently by extractingand storing in a new image file a determined subset of the data in theimage file and deleting the original image file. Alternately, data in animage file may in some implementations be deleted solely by updating thebookkeeping information for the file, without moving any of the imagedata.” And again, the key difficulties in handling of video data remainunrecognized and un-addressed in this type of system.

Castor describes a video image management system in which similar“wavelet or wavelet-like transforms” are applied to “each set of N(e.g., sixteen) successive images (i.e., frames).” More particularly “Inall embodiments, the image file (or files) representing the set offrames is stored so as to facilitate the generation of smaller imagefiles with minimal computational resources” (relative to transcoding).While this method eliminates the need for transcoding processing, itdoes not avoid other significant costs involved in continuous media.

The problem is that a video object of more than a few seconds willcontain large numbers of GOFs. Since the layering is computed and storedon a GOF-by-GOF basis, there will be many small sets of low-significancedata scattered throughout the file structure used to store thecompressed video. The reduction requires “extracting a subset of thedata in the set of image data structures and forming a lower qualityversion of the set of image data structures that occupies less space inthe memory device.” This means that the file must be read, restructured,and rewritten, which can be a significant cost for large files, eventhough the costly decompression-recompression steps of transcoding areavoided.

There is evidently no recognition that similar methods might be appliedto audio, but in any case, similar problems may be expected to arisethere as well. MP3 audio and similar formats are, like MPEG video,designed with real-time transmission and play as a primary objective,and thus must store all data equivalent to a “GOF” or small set of audiosamples together within a short “window” period or transmission packetframe. So reducing the size of such an audio file would involve asimilar elimination of low-significance layer data that is scatteredthroughout a large audio file, with similar processing and input/outputcosts.

It should also be noted that there has been attention to maintaining theability to support functionality like that of a Video Cassette Recorder(VCR), such as random access, fast-forward, backward, fast-backward,stop, pause, step-forward, slow-motion, or other “trick” functions whenusing layered data formats. Consistent with the orientation of layeredmethods to transmission rather than storage, this has been in thecontext of data stored at a remote source server site and transmitted inlayers to the recipient site, as in the Lin paper in the IEEE SpecialIssue, not the possibility of local storage at the recipient/playbacksite, such as is typical of a consumer VCR or DVR.

The underlying broad challenge is the difficulty of simultaneouslymeeting the needs of content creation, initial storage at the content ordistribution source, real-time transmission and presentation to apotentially large number of distant viewers or listeners with varyingcommunications facilities and player devices (whether by appointment oron demand, and/or batch transmission and asynchronous viewing), plus,local storage at the receiving site, and deferred presentation of storedcontent. The primary orientation of most work on media compression istoward the needs of real-time transmission and play, and that objectivewill remain critical. However, as the retention of media objects in along-term storage system, library, or archive at the user's site beginsto involve large collections and substantial resources, the problem ofmanaging local storage over the life-cycle of media objects will alsobecome increasingly important.

SUMMARY OF THE INVENTION

One aspect of the present invention includes a method and accompanyingapparatus for managing storage capacity, wherein an object is stored inmachine-readable format having at least a first layer ofhigh-significance data and a second layer of separately-deletablelow-significance data. A condition for deleting the second layer whileretaining the first layer is determined and the second layer isautomatically deleted upon satisfaction of the condition, whereby theobject is still accessible by a user.

According to a second aspect of the invention, a method and apparatusfor managing data storage capacity includes storing a plurality ofobjects in machine readable format having at least a first layer ofhigh-significance data and a second layer of separately-deletablelow-significance data; determining a condition for deleting the secondlayer from each of the plurality of objects while retaining the firstlayer; and deleting the second layer automatically upon satisfaction ofthe condition, whereby the objects are still accessible by a user.

According to a third aspect of the present invention, a method andapparatus for transmitting data includes generating an object andtransmitting the object to a user in a format having at least a firstlayer of high-significance data and a second layer ofseparately-deletable low-significance data.

According to a fourth aspect of the present invention, a method andapparatus for transmitting data includes generating an object andtransmitting the object to a user in a format having plurality ofseparately-deletable layers of data, with each layer being ofprogressively decreasing significance.

According to a fifth aspect of the present invention, a method andapparatus for storing an object includes receiving an object in a formathaving at least a first layer of high-significance data and a secondlayer of separately-deletable low-significance data and storing theobject in a memory device.

According to a sixth aspect of the present invention, a method andapparatus for storing an object includes receiving an object in a formathaving a plurality of separately-deletable layers of data, with eachlayer being of progressively decreasing significance and storing theobject in a memory device.

According to a seventh aspect of the present invention, a method andapparatus for storing metadata related to stored media objects includesidentifying a location of metadata corresponding to an object having oneor more layers of separately-deletable data; deleting at least one ofsaid layers of separately-deletable data; and identifying a change in anavailability of at least one remaining layer, after deleting the atleast one layer.

According to an eighth aspect of the present invention, a method andapparatus for storing an object in a storage facility that may lackavailable space sufficient to store the object includes, prior to anobject storage request, determining a set of storage units used forcurrently stored objects that could be reallocated without degrading anyof said objects beyond specified thresholds; receiving an object storagerequest; storing the object in the pre-determined storage units, inresponse to said request.

According to a ninth aspect of the present invention, a method andapparatus for deleting at least a portion of a layered object thatemploys one of an index structure and a pointer structure to specifymultiple orderings of storage units, includes: determining an order forstorage units for an object; reading the storage units comprising theobject in a predominantly time-based order; and deleting at least aportion of the storage units in a layer order.

According to a further aspect of the present invention, a method andapparatus for storing a layered object includes tracking a plurality oflayers for an object, each layer stored a plurality of storage units;allowing layers to be individually marked as released; and marking astorage unit for deletion when layers stored in it are marked asreleased.

BRIEF DESCRIPTION OF THE DRAWINGS

Further aspects of the instant invention will be more readilyappreciated upon review of the detailed description of the preferredembodiments included below when taken in conjunction with theaccompanying drawings, of which:

FIG. 1 is a block diagram of an exemplary user storage device, such as anetworked computer or media device, and network for implementing certainembodiments of the present invention;

FIG. 2 is a diagram of an exemplary media data format for the network ofFIG. 1;

FIG. 3 is a flow chart of an exemplary process, performed by the userstorage system of FIG. 1, for storing media data according to certainembodiments of the present invention;

FIG. 4 is a flow chart of an exemplary process, performed by the userstorage system of FIG. 1, for progressively deleting media dataaccording to certain embodiments of the present invention;

FIG. 5 is a schematic diagram of an exemplary directory informationdatabase maintained by the user storage system of FIG. 1; and

FIG. 6 is a flow chart of an exemplary process, performed by the userstorage system of FIG. 1, for retrieving media data according to certainembodiments of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention may be described, in various embodiments, as asystem and method for storing media objects in a way that allows for theefficient progressive deletion of low-significance layers in order toreclaim storage space. The storage format and storage management methodsare structured to permit quality/space tradeoffs to be made not only atthe time of initial storage of the object, but at any time thereafterthat a different trade-off is desired, whether because of increased needfor space or decreased value of the object. The format is alsostructured to permit the subsequent reductions or deletions to be madewith minimal processing time and cost.

It would be very desirable if a stored object could be reduced in sizeand quality, but still retained for future use, providing incremental(“graceful”) degradation in order to maximize data storage space. Thiswould be useful in a range of situations:

-   1. Portable cameras and recorders that capture media objects, but    have limited storage and limited means of communication to external    storage systems. Once such a device is filled at whatever selected    quality/compression level was used, objects must be offloaded by    transmission or removal and replacement of the storage media, or by    discarding them, before any further object can be captured.-   2. Portable media players have similar constraints.-   3. Home (or office) entertainment and media storage systems have    storage constraints that are typically less severe, and a variety of    network accessible fixed or removable storage devices may be readily    available, but the total number of objects to be stored may be very    large. These systems may typically become archival in character,    containing large and diverse collections of objects of various    types, ages, and levels of importance. Both the value of these    objects and the availability of storage resources will vary over    time. Thus techniques for managing the use of resources to match    these various and time-varying value parameters and resource    constraints will become increasingly important.

In all of these situations, an effective and convenient way to altercompression levels to meet changing requirements would be very valuable.Furthermore, a consistent way to do this across a variety of file typeswould also be valuable as broad-function media storage systemscontaining varied formats emerge.

A key objective of the method is to avoid the cost of reading,separating the desired and undesired portions, reformatting, andrewriting for an entire object when a progressive deletion is to bemade, and this is particularly challenging for the important case ofcontinuous media. As described previously, this cost is incurred withcurrent methods for storing continuous media because the portions to bediscarded relate to many media segments, such as GOFs or windows, andthus are in many locations throughout the stored form of the object. Theproposed method is based in part on an alternative storage structurethat groups the layer data so that the parts of the one or more layersthat are to be deleted together are stored together, and apart fromother layers. It is preferably further based in part on matching theexpected deletion actions to file system units, so that the data for oneor more layers that are to be deleted as a unit are stored using thekind of storage unit, such as a file, file segment, block, orsuper-block, that can be deleted by the file system in use within thestorage system without reading and rewriting a larger unit. (Whereexamples are given referring to “block,” they are meant to apply to anysuch storage unit.)

The proposed method exploits the fact that, in well-structured filesystems, much of the metadata that defines what data is stored where andin what format, such as directories of files and file allocation tablesthat track used and free storage blocks, may be stored externally to thedata objects. When this metadata is stored externally to an objectrecognized by the file system, such as a block of storage, it can bechanged without change to the object. Thus the object need not be reador rewritten to make such a change. The external data for managing thefiles may also be maintained in primary storage (a higher level in astorage hierarchy), so that secondary storage (lower level) file systemchanges can be made with no reading and writing at all, except for thelimited writing of updates needed to protect against corruption in theevent of a system failure. For example, in such a file system, a blockof data can be deleted by simply removing any file directory entry thatpoints to it, and marking the file allocation table entry for that blockto indicate that it is unused. The old data may actually remainuntouched until it is reallocated and overwritten with new data.

Another characteristic of file systems that may preferably be exploitedto serve the needs of progressive deletion is that such metadata-baseddeletions may also be undone at a later time, up until the time the dataunit is actually reassigned and overwritten by other data. Thus, deletedspace and the corresponding content descriptors would be marked as such,but would still be potentially recoverable.

The restructuring of data effectively results in the re-ordering of datafor the media object from having data for all layers of each GOF orother short segment together to having all data within a layer together.This is done without changing the data within any layer in any frame orGOF. This method is primarily described here using the example of video,such as might be coded in MPEG, but it will be clear to one skilled inthe art that equivalent methods can be applied to other formats and toaudio, object-based media, and other continuous media content. Also, asnoted herein, and otherwise as will be apparent to one skilled in theart, many aspects of these methods are also applicable to non-continuousmedia, such as still images or other objects that may be coded inscalable or layered form.

It should be noted that a limited embodiment would be to perform simpletruncations of data without regard to significance. One such embodimentwould truncate continuous media objects in time, by simply eliminatingentire time segments, such as by cutting off the end. An alternativesuch embodiment would truncate objects in space, such as by cropping theimages of a video to remove entire areas of image, such as at theborders (like a movie cropped to fit a TV screen). Such embodimentsmight be more desirable than complete deletion of an object, but wouldnormally be expected to be less desirable than the preferredsignificance-based methods in that such non-significance-basedtruncations would very noticeably alter the perceived presentation ofthe object, and be likely to lose critical portions.

As used herein, the term “continuous media” is meant of refer to anyrepresentation of “content” elements that continue and may change overtime, including one or more of “audio data,” “video data,” animation,virtual reality data, hybrid natural and synthetic video data, includingboth “stored formats” and “streams” or streaming transmission formats,and further including “continuous hypermedia” which contain both simplecontinuous media and hyperlinks. Continuous media may containdescriptive metadata, time codes (such as in Society of Motion PictureTelevision Engineers (SMPTE) coding), and other metadata. This usage ismeant to be generally consistent with the definitions contained in theW3C SMIL 2.0 specification, which defines “continuous media” as “audiofile, video file or other media for which there is a measurable andwell-understood duration,” in contrast to “discrete media” as “imagefile, text file or other media which has no obvious duration.”

“Video data” refers to all forms of moving-images, with or withoutaccompanying sound, including digitally coded video, television, film,animation, virtual reality data, hybrid natural and synthetic videodata, and the like. Video image data is most commonly represented as aseries of still images, whether in analog or digitally coded forms,including ATSC (American Television Systems Committee), NTSC (NationalTelevision Systems Committee), PAL (Phase Alternate Line)/SECAM(Sequential Couleur avec Memoire), DTV (Digital TV), HDTV (HighDefinition TV), MPEG (MPEG-1, MPEG-2, and MPEG-4, and supplemented byMPEG-7 and MPEG-21, and other variations), DVB (Digital VideoBroadcasting), International Telecommunications Union H.26x and H.32x,RTP (Real-Time Transport Protocol), RTSP (Real Time Streaming Protocol),SMIL (Synchronized Multimedia Integration Language), QUICKTIME, WINDOWSMEDIA, and REALMEDIA, and the like, but may also be coded as objectdata, including formats provided for in MPEG-4.

“Audio data” refers to all stored forms of sound, whether part of avideo form or not, including digitally coded sound or music or otheraudio information in formats such as PCM (Pulse Code Modulation),CD-AUDIO, MP3, REALAUDIO, MIDI (Musical Instrument Digital Interface),and the like. Audio data is most commonly represented as amplitude dataover time, whether in analog or digitally coded form, although objectdata can also be represented, such as using MIDI.

Animation or virtual reality data is commonly represented in variousimage-like forms, raster or vector graphic forms, or as object-basedstructures, such as scene graphs, including SHOCKWAVE FLASH (includingSWF and Open SWF), SVG (Scalable Vector Graphics), VRML (Virtual RealityModeling Language), RM3D (Rich Media 3D), and MPEG-4/BIFS (Binary Formatfor Scenes), Computer Aided Design (CAD) or wireframe animation, and thelike.

Another important media content type is still images, includingphotographs, drawings, cartoons, diagrams and facsimiles, which may becoded in such formats as JPEG (Joint Photographic Experts Group)/JFIF(JPEG File Interchange Format), GIF (Graphic Interchange Format), TIFF(Tagged Image File Format), PTP (Picture Transfer Protocol), includingobject formats such as CAD and the other object formats listed above,and the like.

A further common media content type is text, which may be coded in suchformats as ASCII (American Standard Code for Information Interchange),HTML (Hypertext Markup Language), DHTML (Dynamic HTML), XHTML(eXtensible HTML), PDF (Portable Document Format), SGML (StructuredGeneralized Markup Language) word processing formats, and the like.Other media content includes active formats, such as spreadsheets, forexample.

“Media content” is used herein to refer generally to any content, orinformation that is understandable to humans. “Content” refers to anyform of transmitted or stored information. “Objects,” when used in thecontext of stored content objects refers to any content item or elementor grouping of items or elements, including objects within a file, andobjects stored as files or sets of files. When used in the context ofobject-based media formats, the term is meant herein to be used inaccordance with the definitions applicable to such formats.

“Storage” as used herein is meant to refer to the process of storinginformation or content for future use, or to any memory, “storagedevice” or “storage system.” “Storage system” refers to any device orany combination of one or more devices with software that supports theuse of storage, including Storage Area Networks (SANs). “Storage device”refers to the element or elements of a storage system that includeactual fixed or removable “storage media” capable of retaining contentin an electromagnetic or other machine-readable form using anytechnology, including electronic, magnetic, optical, time-delay,molecular, atomic, quantum, transmission-delay and the like, includingall future storage technologies. Unless otherwise indicated or clear incontext, “electronic storage” and “electromagnetic storage” are meant toinclude all such technologies without distinction. Storage systems maybe embedded into specific media devices, such as cameras, DVRs (alsocalled Personal Video Recorders or PVRs), televisions (TVs), cable orsatellite set-top boxes, music players, and the like, or may take theform of a general purpose storage system for use with varied contenttypes, both media and otherwise, and with varied media or other devices,such as home or office gateways, storage server systems, or “archiving”systems. “Archiving” refers to the retention of collections of objectsover extended periods, and to associated methods for managing theobjects and their storage and use. Unless otherwise indicated or clearin context, archival is used as inclusive of all components of a storagesystem that may have archival functions, including file servers, and notlimited to components specific to long term storage.

“Storage time” may refer to the time at which an object is stored or tothe duration of time needed to complete a storage operation. “Memory”may be used synonymously with storage. “Hierarchy” relates to the linkeduse of multiple storage technologies having complementaryperformance/cost characteristics, such as high-speed memory caches, harddisks, and low-cost removable media, and the related control methods ofhierarchical storage management (HSM), including those used in priorenterprise data center hierarchical storage systems.

“Transmission” as used herein is meant to refer to any form of“communication” or “transport,” including directly attached devices inoperative communication, local area networks (LANs) including home andoffice networks, and wide area networks (WANs). Transmission may be overany suitable medium, including the Internet and World Wide Web, cableand wireline networks, ATM (Asynchronous Transfer Mode) networks,fiber-optic networks including use of SONET (Synchronous OpticalNetwork), satellite and terrestrial fixed and mobile wireless networks.Local network technologies may also be used, including wired or wirelessLANs and HANs (Home Area Networks). While normally intended to moveinformation from one place or device to another, transmission may alsobe used as a means of storage. Transmission protocols may include IP(Internet Protocol), TCP (Transmission Control Protocol), UDP (UserDatagram Protocol), SCTP (Stream Control Transmission Protocol), RTP,RTCP (RTP Control Protocol), RSTP, IP Multicast, ASF (Advanced StreamingFormat), ATM, Ethernet, GSM (Global System for Mobile Communications)and similar wireless protocols, DSM-CC (Digital Storage Media-Commandand Control), DMIF (Delivery Multimedia Integration Framework), and manyother current and future protocols, and use either baseband or broadbandsignaling. Transmission is typically to a network node address, examplesof which are IP addresses, and logical addresses, such as URLs(Universal Resource Locators) and URIs (Universal ResourceIdentifiers)/URNs (Universal Resource Names). Transmission may becharacterized by bit rate and Quality of Service (QoS), as well as manyother parameters. Unless otherwise indicated or clear in context,transmission is meant to include physical transport of storage media.

“Compression” as used herein is meant to refer to any form ofinformation representation or “coding,” whether for storage ortransmission, intended to reduce the resources needed to store ortransmit the information.

“Coding” refers generally to any type of data representation, and may beused to refer implicitly to codings that achieve compression, such asHuffman or entropy codings. In digital formats the resources typicallyrelate to the number of bits or bytes needed to store or transmit theobject, and reductions are typically achieved by various transforms thatconvert data into more efficient codings. All references to binaryformats and bits are meant to be inclusive of alternative data codingsthat employ multiple valued devices and logics, as well as transmissionsusing multiple signaling elements.

“Decompression” and “Decoding” refer to the reverse of compression orcoding, typically by use of inverse processing or transforms. “Lossy”compression refers to the use of processing in which the inverse doesnot fully recover all information from the original, whether the data islost by explicit discarding or by implicit transformations, and“lossless” refers to compression that does not lose such data.

“Transcoding” refers to any transformation from one coded format toanother that involves decoding and recoding, which may include suchsteps as resampling, mathematical transforms (such as discrete cosinetransforms, wavelet or wavelet-like coding, and fractals), quantizationand entropy coding.

“Layering” as used herein is meant to refer to the ordering and/orseparation and/or grouping of compressed data based on the“significance” or importance of the data to the quality of perception ofthe decompressed result. Data is separated into a base layer (BL) andone or more enhancement layers (EL). The characterization of coding as“scalable” or “embedded” are understood as generally equivalent tolayering for the purposes addressed herein. Layering may be based onframes or samples or sampling rates or bit-depths or other aspects ofcoding, which can vary quality in terms of such factors as resolution orfidelity, and may involve discrete layers stored as distinct elements orto simple orderings or logical separation of layer data within anobject. “Significance” may refer to either or both of mathematicalsignificance or subjective perceptual significance. “High-order” or“low-order” may be used synonymously with “high-significance” or“low-significance,” respectively. Scalability may relate to multiplecontent attributes, including temporal, spatial, and quality. “FineGranular Scalability” (FGS), refers to the scheme by that name proposedby Radha, and similar methods, and a variation on this is ProgressiveFine Granularity Scalability (PFGS). Depending on context, “quality” asused herein may relate to objective measures such assignal-to-noise-ratio (SNR) or to subjective measures of perceivedquality or fidelity, including both broad and specific metrics. Qualitymay be specified in terms of equivalent or related metrics, includingnumber of layers, amount of space, fraction of space, amount of quality,fraction of quality, arbitrary quality units or “quality factors” andthe like. Unless indicated otherwise, examples given in terms of onesuch metric are meant to include alternative metrics as well. As thebenefits of techniques such as those disclosed here become understood,it may be expected that the use of scalable or layered formats willspread and will come to include media types which are not commonly codedin scalable forms at present.

“Progressive” refers to coding schemes that may layer data forsuccessive transmission and presentation at progressively improvingquality, or to the progressive deletion of one or more of said layers asdescribed herein. Unless otherwise indicated or clear in context,“progressive” as used herein does not refer to interlacing techniques,such as the physical screen raster scan sequence distinction ofprogressive versus interlaced scans.

“Progressive deletion” is meant to refer to the discarding of data in aprogressive manner, allowing for one or more deletions of data whileretaining sufficient data to maintain a reduced quality result, in aprocess that results in “graceful degradation” (much as the term usuallyrefers to degradation of a system in such a manner that it continues tooperate, but provides a reduced level of service rather than failingcompletely) with regard to the quality of the object over time or asavailable data storage space decreases. Progressive deletion typicallydiscards low-significance data in preference to high-significance data.The data ordering suited to progressive deletion may be referred toherein as “archival order.”

A “group of frames” or “GOF” refers to a number of frames (or pictures)that are treated together for effective representation of inter-framedifferences, such as due to motion. Such GOFs are typically on the orderof 16 frames, but larger GOFs may be used by more advanced algorithms oras processing economies improve. A GOF is typically short relative tothe duration of an extended video sequence. The frames within a GOFtypically include key frames (or I-frames) and non-key frames, includingthose (P-frames) that are coded in reference to a previous frame andthose (B-frames) that are coded to be bi-directionally predicted orinterpolated. A “window” refers to a set of samples that are treatedtogether in coding processing, and thus generally analogous to a GOF,such as for audio data, which may consist of a series of amplitudesamples. Codings may also address inter-GOF or inter-windowrelationships. A “media segment” refers to any time-continuous portionof a continuous media object.

“File” as used herein is meant to refer to generally to a media or dataobject, but may more particularly refer to a discrete object as handledby a storage system. “Stream” also refers generally to a media or dataobject, but may more particularly refers to a technique for transferringdata such that it can be processed as a steady and continuous stream.“File” and “stream” may be used here interchangeably except where a morespecific meaning is appropriate. Streaming is commonly used for thetransmission of continuous media. This may be in a format intended onlyfor real-time presentation that is not intended to be stored, but mayalso include formats that may be stored for future use at the receivingsite, and such streams may be contained in a file and advanced filesystems may provide for support of streams as objects within files.Continuous media may be transmitted as a stream, and individual layersmay each be transmitted as a stream. Continuous media may also be storedas a Binary Large Object (BLOB). “File structure” is used to refer toschemes for structuring data within a file, such as into streams orobjects, as well as for structuring data across groups of files. “Filestructure” may also refer to the structuring of groups of filesthemselves, and the “metadata” that describes those files, includingsuch metadata structures as file tables and directories, File AllocationTables (FATs), inodes, and indexes. “File structure element” refers to agrouping of data elements possibly including attendant metadata,including, but not limited to those that may be made to correspond to aphysical storage unit, such as a file, file segment, block, segment,cluster, extent, or super-block. “Storage unit” is meant to refer to anysuch unit that is readily manipulated. “Input/output,” “inputoperation,” and “output operation” refer to physical reading or writingto or from a storage device. “File system” refers to the softwarefacilities that support the use of files and file structures, whetherstandard operating system facilities or a supplementary package or builtinto a specialized systems or application software system, and is meantto include virtual file systems. File, file system, and file structureare also meant to be inclusive of “data base systems,” includingrelational data bases, object data bases, and object-relational databases, and inclusive of “directory systems,” including those based onLDAP (Lightweight Directory Access Protocol).

“Deletion” as used herein may refer generally to the removal or erasureof data from all or part of an object. Deletion usually entailsattendant reduction and release of storage space used by the object.More particularly, in the context of the stored form of an object withregard to a storage system and the file structure elements and storageunits that contain it, “deletion” refers to a deletion of one or morecontiguous elements without removing any of the remaining data, such asmight preferably be done at the file structure or storage system commandlevel and result in the release of the associated physical storagespace. In this context, “erase” may be used synonymously with delete.

“Metadata” refers to data about data, including descriptors of datacontent and of data format and “program information.” Metadata formats(that are general purpose or related to media content) include XML(eXtensible Markup Language), RDF (Resource Description Framework), SDP(Session Description Protocol), SAP (Session Announcement Protocol),MIME (Multipurpose Internet Mail Extensions), MPEG-7, MPEG-21, SMPTEtime codes, ATSC-PSIP (ATSC-Program Service Integration Protocol),DVB-SI (Digital Video Broadcast-Service Information), and SMIL, as wellas data contained in Electronic Program Guides (EPGs).

“Multicast” as used herein is meant to refer to the transmission of datato a defined group of recipients. Internet multicast protocols, such assupported by the Internet Multicast Backbone (MBone) and IP Multicast,provide for this in the form of a stream or channel to which recipientsmay subscribe. Multiple related multicast streams may carry relateddata, such as in the case of a RLP “layered multicast,” where a baselayer channel and some number of enhancement layer channels containpartitions of the layered content data.

When used herein to describe entertainment system elements, “system” mayrefer to all elements of the entire network from capture or creation topresentation, including elements at many sites, or to elements at onestage or one site, or to individual subsystems. “Subsystem” is meant torefer to elements that are typically specific to some function or task,but such elements may also serve as a complete system. Usage of “system”and “subsystem” and similar terms may suggest typical differences, butare not meant to be limiting, and such terms may generally beequivalent, especially given the broadening usages and convergences oftechnology and products applicable to this field.

“Server” and “device” are also used as equivalent to system orsubsystem. “Server” may be suggestive of elements that may provideservices to other elements, possibly on a shared basis, and “device” maybe suggestive of packaging as a discrete hardware component or product.Future systems and subsystems may include components and services thatare provided in software form, much like PC applications.

“Consumer,” “user,” and “end-user” as used herein are meant to begenerally equivalent in referring to individuals, including small groupsof family members or of office workers, especially those who frequentlyshare a common base location at which a user system might be located.

“User system” as used herein is meant to include any kind of system thatmight be usable for reception, storage, and/or presentation of mediacontent by users of such content.

The specific formats, standards, protocols, algorithms, and the likelisted here are meant to be exemplary, and applications to other similarformats, standards, protocols, algorithms, and the like are intended.Where examples are given in terms of one class of media, system, format,and the like, they are meant to be representative of equivalent casesfor other applicable classes.

Referring now to FIGS. 1-7, wherein similar components are referenced inlike manner, various features for a method and apparatus forprogressively deleting media objects from storage are disclosed.

Turning now to FIG. 1, there is depicted an exemplary computer or mediadevice network 100, which includes a transmission network 105 providingtransmission services through which one or more user systems 114 havingsub-systems for reception 107, storage 109, and presentation 111 maycommunicate with one or more media source systems 113 having subsystemsfor capture/creation 101 and production/distribution 103. Each of thesubsystems for reception 107, storage 109, presentation 111,capture/creation 101 and production/distribution 104 may havecorresponding storage subsystems for storage of associated mediacontent. The detailed structure of these subsystems is not essential,and may involve finer breakdowns or specialized subsystems, such as forthe separate common storage subsystem 109, with its storage devices 110that serves both reception and presentation subsystems, or combinationsof the subsystems shown here into combined elements, such as forreception, storage, and presentation, such as current DVRs. Forsimplicity, the various subsystems 101, 103, 107, 109, and 111 will beconsidered to include any associated storage devices 102, 104, 108, 110,and 112 respectively, except where the context indicates otherwise.

Various elements of the user system 114 may be embodied in the widerange of current and future devices for use of media, such as set-topboxes, DVRs/PVRs, advanced/digital TVs, PCs, PC-DTV (Digital TV)devices, audio systems and recording devices, home entertainmentsystems, game systems, telephones/videophones, home gateways, portableaudio/video/media/game devices, including Personal Digital Assistants(PDAs), telephones, music players, remote controls, and server devicesthat perform such functions, both individually and in combinedconfigurations. The methods described here apply to all such current andfuture variations in the configuration of the basic functionalcomponents described here with minor variations that will be apparent tothose skilled in the art.

Preferably, advanced user systems would make use of a common mediastorage and archiving system, as a function-rich embodiment of storagesystem 109, that takes content objects as they are received by one ormore reception subsystems, maintains them using one or more storagedevices, and makes them available to one or more presentationsubsystems. Such a configuration might minimize or eliminate need forseparate storage devices for reception 108 or presentation 112, andmight provide for greater economy and functional richness.

The user system will preferably serve co-located users at a home oroffice, and thus avoid the constraints, complexities and costs thatusually relate to wide area transmission. Remote access may also beprovided. Such access would preferably be handled as secondary, as ameans to support remote use, including access to stored media, by userswho may be temporarily away from the base location, as well assupporting some remotely-based users on a supplementary or occasionalbasis. The user system might also be portable or mobile, includingwearable systems.

The elements just described may use any technology suitable for thehandling, storage, and communication of media content, include allnetwork and transmission technologies currently used for media, andpresumably all future such technologies. Network 105 may preferablyinclude a combination of available methods, including all transmissiontechnologies listed above. The various storage devices listed mayinclude any suitable technology, including all those listed above. Theuser system 114 and its component reception 107, storage 109, andpresentation 111 subsystems may include any device available toconsumers (or business end-users) providing such functions. Such deviceswill include the full range of components suited to such devices,including CPUs and other processors, clocks, various specialized logicprocessors, including, CODECs (Coder/Decoders), DSPs (Digital SignalProcessors), ASICs (Application Specific Integrated Circuits), caches,RAM (Random Access Memory), ROM (Read-Only Memory), and other memory andstorage devices, including the various forms listed above, buses andconnectors, various transducers for local and remote communications anddevice interfacing, including radio frequency (RF), Infra-Red (IR),fiber, coaxial cable, telephone cable, multiplexors/demultiplexors, andmodems or other analog-to-digital converters. Such systems andsubsystems will also employ any suitable software technologies andoperating systems, including computer operating systems such as WINDOWS,UNIX and LINUX, embedded operating systems such as WINDRIVER VXWORKS,MICROWARE OS-9 and DAVID, as well as other system software such as JINI,Web servers, Web services, agent systems, and programming languages andenvironments, such as JAVA, C, C++, C#, J2ME, and the like. Standardfile systems and database management systems may also be employed.

Media sources may include any suitable source of content, such as forexample broadcast television, cable television, satellite television,digital television, Internet broadcasts or multicasts, World Wide Web,digital video discs (DVD), still images, video cameras, laser discs,magnetic media, computer hard drive, video tape, audio tape, dataservices, radio broadcasts, or any other form of communications ordistribution. It should also be understood that there may be variationsin sourcing. For example, the source may be local to the user system,such as in the case of a camera, thus requiring only local transmissionamong sub-systems. Also, for example, the content may be obtained on aphysical medium, such as videotape or DVD, which also can be thought ofas substantially equivalent to a local source. Media source systems 113and their component capture/creation 101 and production/distribution 103systems may be of any suitable technology and configuration, includingall of the alternatives described above for user systems 114, and mayinclude hardware and software configurations suitable for personal/homeuse, as well as server systems, including large scale configurations formedia production and serving, as are well known in the art.

Although the embodiment described herein involves components of typicalmedia systems and computers and network servers, other existing orfuture technologies that perform similar functions may be employed. Onesuch variation is the use of so-called “web services” or otherdistributed service technologies in which functions typically performedby a single server complex operated by a single enterprise may be“distributed” so as to integrate component services provided on remoteservers operated by independent enterprises into a cohesive “virtualserver” offered by the combined “virtual enterprise.” A similarvariation is the use of “application service providers” (ASPs) tooutsource services, whether personal or business.

Also clearly intended is the use of multiple cooperating servers, aswell as the use of multiple cooperating client systems and peer-to-peerarchitectures, as well as the use of mobile agent technologies.Variations may include assemblages based on combinations of downloadableprograms, plug-ins, applets, aglets, or other distributed components andthe use of removable components such as smart cards. Future embodimentsof media source and user systems may be based on a wide spectrum ofintelligent devices including cell phones, PDAs, wearable computers andsensors, and the like, and may involve mobile applications that movefrom device to device, as needed.

It is also to be understood that while the discussion herein is in termsof conventional electronic digital computer systems, future equivalenttechnologies might also be used. Such alternative technologies mightinclude optical, photonic, quantum, molecular, or organic computingsystems, and the like. Accordingly, references herein to electronictechnologies are meant to be inclusive of embodiments based on suchfuture technologies as well.

System elements will preferably conform to formal or de-facto standards,such as OpenCable, Open Services Gateway Initiative (OSGi), UniversalPlug and Play (UPnP), Home Audio/Video Interoperability (HAVi), VideoElectronics Standards Association (VESA), Architecture for HomeGate(AHRG), AUTOHAN, MHP (Multimedia Home Platform), DASE (Digital TVApplications Software Environment), and the like. Digital RightsManagement (DRM) technologies may be provided, including devices fordecryption and for identification of users and their entitlements,including OpenCable Point-of-Deployment (POD) modules, smart cards, orothers.

Turning now to FIG. 2, displayed therein is depicted an exemplarystorage structure of a continuous media object 200. The object isdivided into n frames or samples S1, S2, . . . Sn, that representelements of content at moments in time These frames or samples aregrouped into defined segments, such as GOFs or windows, shown as GOF1,GOF2, . . . , etc. This horizontal structure follows the inherenttime-based structure of the media content, corresponding to time runningfrom left to right (as diagrammed here), that applies to bothtransmission and presentation. Standard compression techniques deal withthese elements, typically compressing each GOF independently, butoptionally adding some data on GOF to GOF changes.

The layering of the data is shown in the vertical dimension, with a highsignificance base layer L1, and additional enhancement layers L2 . . .Lm of successively lower significance. Layer L1 is essential topresentation, but all other layers are supplementary. Depending on thespecific compression method, data would typically be layered on a frameby frame basis, with successive layers for frame 1 shown here as S(1,1),S(1,2), . . . S(1,m).

Many compression algorithms apply motion compensation across the frameswithin each GOF. In this case the data for the initial or key frame of aGOF will usually be complete, but data for other frames (i.e,enhancement frames) within a GOF may be reduced to contain onlydifference data relative to the key frame. This might apply to both BLand EL data. Thus the data for frame S2, and other non-key frames withinGOF1 may not be usable without also having the data for frame S1.

To provide a simplified but more concrete example (which may differ indetails from any actual media content), for video there might be a videomotion rate of 30 frames per second. Each frame might average 100Kbytes, and if layered into ten layers, the data for a layer of a singleframe, S(i,j) might average 10K. A 16 frame GOF might then contain1,600K bytes. Data for key frames would normally be more voluminous thanfor non-key frames.

In a conventional layered transmission, data would be sent in accordwith the horizontally depicted time dimension, since that permitsreal-time presentation. In real-time presentation, all data layers to bepresented for a given frame must be received before that frame ispresented, or they will no longer be of use. Various alternatives, suchas the motion compression just described, may involve interdependentgroupings of data across frames, but together within GOFs or other smallsegments. For non-key frames, such as for example for frame S2, much ofthe data may be predicted from data for the preceding key frame (S1), soonly a few motion adjustments need be transmitted. In any case, all datafor a relatively short segment of time must be transmitted closetogether in time to allow for real-time presentation. Thus, data to betransmitted is stored in this left to right order, at least at the GOFor segment level, and in conventional systems such data is stored on thereceiving end in the same order.

In such conventional modes of storage, each GOF or window contains alldata for some segment of time and so the desired top layer data isscattered throughout the object, and cannot be deleted without readingthe object, reformatting it to remove the one or more top layers, andrewriting the resulting smaller object. This may involve one or morestorage blocks or other units, but still crosses portions of all theunits (samples, frames, GOFs, windows, segments, etc.) that comprise theobject

What is needed in order to enable progressive deletion is to store thedata in a layer-by-layer sequence, so that an entire layer can bedeleted in the fewest possible operations. Following FIG. 2, thisreorders the data into a bottom-to-top vertical order, going from Layer1 for all frames in an extended segment, to Layer 2, and in turn, toLayer m. In such a structure, all data for a single layer Lm or for somenumber of multiple layers Lm, Lm-1, . . . can be efficiently deleted.

Preferably each layer is stored in a separate file, or storage block, orother storage unit that is readily deleted using the facilities of thefile system in which the data is stored. It is further preferable thatthat unit be of a kind, such as a file, that is managed in terms ofmetadata, such as file allocation tables, that can be deleted by thefile system merely by changing the metadata to mark it as deleted,without need to read and rewrite any of the actual data. Similarly, anydirectory structures that are used to relate the component portions of adata object (whether external or internal to the file system) may alsobe structured so that deletion of one or more layers does not requirereading and rewriting of the object. This can be accomplished in avariety of ways that will be apparent to one skilled in the art afterreview of the discussion herein, such as by using separate directorymetadata files structures that are external to the content objects.Alternatively, if such directory data is to be stored within the object,it may be structured so that at most one read/write operation is neededto update the layer directory information. In large objects composed ofmultiple storage units, this can be done in a variety of ways, such asby placing all such data in one block (such as the Layer 1 block), or bychaining the data so that only one block in the last layer retained needbe updated to indicate that no further layers exist. Such methods canpreferably be forgiving of losses of access to directory data orincomplete updating of chains stored with the object (whether byfailures or by export of incomplete data) by using chaining ordirectories of layers that run from high to low significance, and use asmany layers as can be found, assuming that any additional lowsignificance layers not found have been deleted.

It should be noted that a layer need not be stored as a single file orother storage unit, but depending on the size of the content object andthe desirable parameters of the storage system, may be broken intomultiple files or storage units. Alternatively, with appropriatestructures and facilities for efficiently managing storage and deletion,as described below, the multiple layers may be stored within a singlefile unit. Also, the orderings described here need not always correspondto physical order on a storage device. If the storage system providesfor efficient mappings of logical data structures to physical datastructures, it sufficient that the logical data structure be provided asdescribed. Thus, a storage system may permit logical groupings, such asindex lists of blocks or block pointers that facilitate scatter readsand writes, and/or bulk deletion of all blocks for a given layer. Inthis case the reordering described here may be accomplished by logicalmanipulation of pointers rather than physical reordering, and this maybe done by creating directory entries that specify the sequence of allblocks that make up a given layer.

Referring now to FIG. 3, therein is depicted an exemplary process 300 bywhich a reception device 107 or associated storage system 109 mayconvert a media object received in conventional format to the storageformat suited to progressive deletion, such as that in FIG. 2. Theprocess begins with the receipt of a media content stream intransmission format (step 302). The input stream is processed to isolateeach frame or sample in turn (step 304), and to further extract eachdata layer (step 306). The output storage structure is then created byoutputting the data for each layer into separate files or storage units(step 308), using buffering techniques as appropriate. This processpreferably continues until all frames or samples have been processed(step 310). Upon completion of all of the frames or samples, directoryentries are recorded and the resultant output files are closed (step312).

During this process, the output data may be collected in buffers as eachframe or sample is processed to avoid unnecessary input/output activity.Actual writing of file outputs to the storage device would preferably bedone only at the completion of a data block or when the data for theobject is complete. The result is a stored format that is structured tofacilitate progressive deletion.

Turning now to FIG. 4, therein is depicted an exemplary process 400 bywhich a progressive deletion action is effected. The process begins inresponse to a request to perform a progressive deletion action, andstarts with a determination of how many layers are to be deleted, basedon the size or quality reduction desired (step 402). Next is required adetermination of which one or more storage units are to be deleted,which may involve the lookup of relevant details in a directory (step404). The directory may be internal or external to the stored form ofthe object, as outlined previously. This example assumes it is external.Commands are then issued to the file system to effect deletion, whichpreferably is done by marking the deleted storage units as free (step406). Finally relevant directories are updated to correspond to thereduced version of the object (step 408). The process is repeated ifadditional progressive deletion actions have been requested (step 410).The result is the progressive deletion of a low-significance portion ofthe subject media content object.

Referring now to FIG. 5, therein is depicted an exemplary directorystructure 500 that may preferably be used to track the relevant metadatadescribing stored media content objects that are managed by a storagesubsystem which provides for progressive deletion. Such a structure maycontain the information needed to perform progressive deletion, as wellas information for a wide variety of other storage and archivemanagement tasks. Each media content object is shown as having one ormore row entries in this structure, one for each layer that is stored.For each such entry, the columns show data elements that may preferablybe maintained. Each content item may be identified by an item Name field502, an item Type field 504 which may specify the nature of the item,and a Description field 506, which may include data indicating theobject's source, content, subject, and format. More precisespecifications of the data stored in Coding Format field 508 may beincluded to control processing. Each layer may be further described byLayer number field 510, which may also contain an indication of whichlayer is the last maintained (shown here as a “*”). Layer size field512, may be in such units as bytes or blocks. Layer quality field 514,may be in any scale, such as a percentage or an arbitrary quality level.Layer filename field 516 may point to the file or other storage unit ofobject content. Finally, Status field 518 may indicate if the layer issaved or deleted (and may also indicate which layer was the lastmaintained). A directory structure of this kind conveniently organizesthe basic information needed to support progressive deletion, and canalso provide the base for a wide variety of additional functions relatedto the management of an archive of media content for a user or group ofusers, in either a home or business context. Preferably relevantstandards for multimedia metadata, such as MPEG-7, will add support formetadata supportive of progressive deletion, such as the elements justdescribed. It is to be understood that such a directory structure mayinclude the data itself or include links to the data, and that it may bestored in a file structure, database system, or directory system, andthat it may be distributed across multiple sub-structures, sub-systemsand storage devices, as desired. It should further be understood thatsuch a directory structure may desirably be integrated with other kindsof directory data and metadata relating to the media objects.

Turning now to FIG. 6, therein is depicted an exemplary process 600 bywhich a presentation device 111, or associated storage system 109, mayretrieve and present a media object retained in the storage formatsuited to progressive deletion. Using any necessary directories or othermetadata, such as described in FIG. 5, the playback system selects allavailable retained files that are appropriate for presentation of thedesired object (step 602), preferably ignoring any layer data not suitedto the desired playback system. For each current frame or sample inturn, the corresponding data from each such layer is read (step 604),and combined into a stream segment of the usual format (step 606). Itwill be understood that just as these methods for reordering datamaintain and re-establish time-based relationships at the frame orsample level in accord with the steps described here, similarrelationships at the GOF or window (or other segment) level that relateto inter-frame or inter-sample data will be similarly maintained. Thedata is then presented in time sequence (step 608). This process repeatsuntil presentation completes (step 610), after which the files areclosed (step 612), and any housekeeping tasks are completed. Againbuffering would be used as appropriate in reading files and in passingdata from storage system to presentation system components.

Preferably the playback system would support VCR-like functionality,such as random access, fast-forward, backward, fast-backward, stop,pause, step-forward, slow-motion, etc. It will be apparent that thesefunctions can be provided in accord with the proposed layered storagestructure using steps similar to those just described to coordinate theretrieval and combination of the multiple layers of data to reestablishtime-based relationships.

Specific formats for the coding of media streams may vary in the detailsof their structure from the example described. One such variation maydepend on such factors as the use of quality/SNR, temporal, or spatialscalability. For example temporal scalability may allow the presentationframe rate to vary, including key frames (as base frames) in the baselayer, but treating all data in non-key frames (or enhancement frames)as enhancement data that may be dropped entirely. Another variation mayrelate to the use of object-based codings and associated metadata, whichmay also be scalable and layerable, but in this case at an object level,such as may be described by metadata when an object first appears (orundergoes a significant change), rather than at a frame or GOF or evensegment level. The need for and methods of the reordering will besubstantially as described here, relating to the shift from a primarilytime-oriented ordering suited to realtime transmission and presentationto a primarily layer-oriented ordering suited to progressive deletion,with appropriate minor variations as will be apparent to one skilled inthe art. Integration with any DRM systems would preferably be providedusing the methods usual for other media storage and retrieval systems.

Numerous variations on and extensions of these methods will be apparentto those skilled in the art and will become practical as the use ofvarious digital content forms develops, becomes more widespread, andconverges onto a compatible suite of devices and systems.

One set of variations will depend on the nature of the media content,and the form of transmission offered. The examples used above dealprimarily with data transmitted without regard to how layers might beused, and without anticipation of the use of progressive deletion at thereceiving end. In such case, all of the actions described to convertsuch content to a format better suited to progressive deletion must beperformed at the receiving end. Preferably media content sources wouldbe aware of the desire to apply progressive deletion and might takemeasures to facilitate storage by the receiver in a desired format.

One set of such embodiments would be to provide a deletion-orientedformat from the source. This might be of limited feasibility for contentthat is transmitted in real time, for the reasons given earlier, butmight be done to varying degrees in some cases. In the case of contentthat is not provided for presentation in real time, but is provided fordownload and subsequent play, there would be no need for the usualreal-time time-sequence order, and the provision of such content in thelayer-by-layer order suited to progressive deletion could be offered.The same would apply to content that is provided by physical delivery onstorage media.

Another such embodiment would apply to the case of transmissions thatemploy layered multicast. As described previously, RLM layered multicasttechniques transmit multiple layers as separate parallel streams suchthat the streams are synchronized to permit a receiver to obtain thenumber of layers desired and to combine them in real time forpresentation at the desired quality level. That synchronization would beclose enough to avoid need for excessive buffer storage at the receiver.Existing layered multicast methods do not consider the possibility thatlayers might be stored for future use, nor do they anticipate that theymight be progressively deleted, but they do separate the layers justprior to transmission in a way that is similar to the format describedhere. To adapt such transmissions to use in a storage system thatpermits progressive deletion as proposed, the receiving system would notcombine them on receipt, as would ordinarily be done for presentation,but instead, each layered stream that is received would be keptseparate, and be stored into separate storage units, using such separatebuffers as might be needed to do that efficiently. This may be doneusing methods similar to those detailed above in the discussion of FIG.3, except that instead of obtaining a single stream and splitting it,the content is received in the form of multiple streams that havealready been split, and the task is one of storing it as described instep 308. Thus buffers may be provided for each layered stream as it isreceived, and writing would proceed as the multiple layer streams arereceived. Once stored in this fashion, the subsequent progressivedeletion actions can be performed in the same manner describedpreviously. Preferably such a system might also be able to combine thestreams in the usual manner as they are received to allow real-timeviewing while recording, in a step additional and concurrent to thestorage steps described here that would provide for storage, deferredviewing, and progressive deletion.

As the use of progressive deletion becomes commonplace, content sourcesmay preferably provide content in forms that facilitate it. For example,the layer data in a media object 200 may be structured to package layerdata to facilitate reformatting and storage in the reordered formdesired for progressive deletion. This may include control of layeringmethods to 1) provide chunks of data of a size and uniformity that iswell suited to storage in reordered form, 2) select layer data sizesthat match to intended storage unit sizes, and 3) use coding formatsthat are well suited to the reordered format. The selection of layernumbers and sizes may be done based on criteria that optimize not onlyfor transmission and playback, but also for reordering and storage inthe layered format suited to progressive deletion. In the case oflayered multicast, similar considerations would be given to layeringschemes that optimize not only transmission but retention andprogressive deletion. In the case where content is to be bothtransmitted in layered multicast and delivered via download or inphysical storage media, the layering parameters would preferably beselected to be suited to both uses.

This balancing of transmission and storage performance objectives mayinvolve many trade-offs. As described, variability in transmissionargues for Fine Granular Scalability, effectively creating a singleenhancement layer with an internal order, while storage managementefficiency may argue for a small number of discrete enhancement layers.The problem is that most storage management systems and storage devicesare not efficiently operated in a fine-grained way. Storage is typicallyorganized into fixed sized blocks for efficiency of I/O operations andspace allocation management. Variable sized blocks andbyte-addressability have been accommodated in some storage systems (diskand flash), but this can result in inefficiency in storage use andperformance. A solution may preferably be to transmit in FGS format, andthen partition that FGS enhancement layer into the number of layerssuited to the storage system at the receiving end, again following themethods described in FIG. 3. Alternatively, more advanced codings, suchas PFGS, which seek to obtain the transmission efficiencies of FGS (finegranular bit-rate scalability, channel adaptation, and error recovery)in a multiple EL format may help address these trade-offs.

The desired number of layers for the stored format would be set based ona combination of the following factors:

-   -   Storage coding efficiency of the resulting content    -   Processing resource issues in coding, decoding, and reordering    -   The layering structure already provided in the content as        received    -   Desired granularity for deletion    -   Basic file system parameters of physical and logical block        sizes, and their performance effects for access and deletion    -   Fragmentation overheads caused by deletions, and the        applicability of defragmentation processes (as well as the        special considerations of any variable block-size        implementation)    -   Use and design of index structures and how they relate to the        block sizes—these might preferably include one index of blocks        for order-of-play across layers and another index for        order-of-deletion by layer    -   The resource demands on the player imposed by the handling of        multiple layers    -   The size of the archival storage unit—a small size limits the        need for transient working storage and buffers for re-ordering,        and a large size allows for most time-efficient deletion (as        well as affecting alternative layer packing strategies and other        efficiency factors in complex ways, as described below).        The specific trade-offs here are essentially engineering issues        that depend on the situation and the details of that will be        apparent to one skilled in the art.

Preferably, especially in cases where media objects are large relativeto storage blocks, indexing schemes can be employed to give the effectof having data stored in both time and layer order. This can be done,for example, by maintaining one time-ordered index of blocks to useduring playback (presentation) to list blocks by order-of-play, forefficient time-based access across layers, and another archival index touse for progressive deletion that specifies order-of-deletion, by layer,for the entire object. The simplicity of handling both kinds of activityin such a scheme may argue for designing media storage systems to havestorage units of a size well suited to such a two-dimensionalstructuring.

Most of the examples described here fully separate the layers inphysical storage units such that (logical) layer boundaries map to(physical) storage unit boundaries without overlap. Such structures aresimple and are well suited to objects that are large relative to thesize of the physical storage units that are released in deletions, butcan allow for less than optimal use of space at any given time byleaving one storage unit per layer partially empty (which can become anissue when storage units are large relative to total storage). Analternative method is to store the partitioned layers in the samelogical layer sequence, but stored with one layer immediately followinganother without aligning layers to storage unit boundaries, so that oneunit may include part of two (or more) different layers. This can avoidwasted storage space, especially when layers are small relative tostorage units (or storage units are large). In such case progressivedeletions would free a storage unit only when all layers of data in thatstorage unit are deleted, and the directory data on layers would includesome mapping data indicating which blocks have which layers, and wouldpreferably specify the offsets of where layer boundaries occur withinblocks that contain multiple layers. In this case, when a progressivedeletion was done, there would be additional steps to determine whichblocks were entirely releasable, and which must be retained ascontaining layer data that was not releasable, and to update thedirectories accordingly.

Allowing for such variations, where a given storage unit may containonly one layer, or, with overlapping storage boundaries, may contain twoadjacent layers, or where multiple small layers may be packed into onestorage unit, the deletion method may preferably be generalized to thefollowing form. The method keeps track of what (or at least how many)logical layers are in each physical storage unit, and as progressivedeletions are done, keeps track of which (or at least how many) layersare to be retained. When the number of retained layers in a storage unitgoes to zero, the storage unit is released. Thus when layers are alignedwith storage units, a single layer deletion releases the containingstorage unit, but when they are not aligned, the release occurs onlywhen all layers in the storage unit are surplussed. In an implementationwhere different objects may share a storage unit, the method would applyacross all objects within a given storage unit.

In the case of non-continuous media, such as still images, for example,or any other form of content that may be scalable, the need to reorderlayer data from time order to layer order as described above does notapply, but the issues of managing the relationship of logical storage oflayers to physical storage remain, and similar methods to those justdescribed apply to this case as well. In particular, the trade-offbetween fully segregated layers and layer stored so that theirboundaries overlap storage units remains, and the design of data anddirectory structures suited to efficient progressive deletion forexpected layer and storage unit sizes applies similar principles andmethods. These methods enable progressive deletion to be done in a waythat is efficient in both processing and storage resources, withtrade-offs that can be tuned to the nature of the media objects and thestorage facilities in the manner just described.

Considering all of these factors, the above-mentioned method ofselecting layer data sizes that match to storage unit sizes can beexploited to make a highly efficient system. This becomes particularlyattractive when transmission considerations do not cause conflicts, andlayer sizes can be selected based primarily with regard to user systemrequirements, such as when content is received in an FGS format,converted from a non-layered source, or directly obtained at the usersystem, such a with a camera (video or still). In this variation of themethod, the parameters that govern the process of layering are set tocause the layers to be of a size that is an exact multiple of thestorage unit size (at least for objects beyond some minimum size).Correspondingly, the storage unit size is preferably engineered to bewell-suited to the typical layer size. This method constrains qualitychoices to some degree, but enables significant economies in storage useand manipulation (and that is the essential goal of compression). Withthis method, layers map neatly to storage units without wasted space,layer directories simply follow this clean mapping, and progressivedeletions involve simple release of some whole number of storage units.This strategy is readily accomplished using coding methods that achievefine-granular scalability (allowing layer boundaries to be drawn asdesired, but can also be done with other coding methods, as will beunderstood by those skilled in the art. This method is particularlyattractive for use with still or video cameras, with their tightlyconstrained memory size, energy use, and performance requirements, andgiven the high cost of storage access and manipulation that puts apremium on efficiency. Camera designers can manipulate all of therelevant parameters to design or select storage with appropriate unitsizes, and to do the layering to match those sizes. With such a methodin a still camera, for example, a “fit one more picture” operation canalways be enabled by 1) always knowing how much space is desired for thenext picture and what storage units can be taken to fit it (by doing thepreparatory processing any time camera picture settings are altered),and 2) simply taking the new picture, coding it, storing it in thedesignated storage units, and revising the directory accordingly, thusimposing little time or energy cost or input/output activity beyond thatof shooting with an empty storage. Also, as described earlier, untilthat next picture is actually taken, the slated deletions remainrecoverable with no data loss.

It will be further understood that the included examples of the methodare described in terms of relatively simple file systems, and thatsimilar methods may be used to exploit advanced file system features tostore and progressively delete layer data in an efficient manner. Forexample file systems that support streams as objects within files couldstore the layers as separate streams within a file to facilitate theirdeletion without interference with the remaining streams. Thus, withsuch techniques, progressive deletion benefits may be obtained even ifany or all of the multiple layers are more efficiently stored andmanipulated within a single file. File systems that support sparsestreams, such as MICROSOFT NTFS, may also be exploitable to use thosefeatures to efficiently delete layers in similar ways. Also exploitableby similar methods is support of fractional blocks or “fragments”.

It will also be understood that a variety means can be used to exploitRAID (Redundant Arrays of Independent Disks) and striping in conjunctionwith the methods described here. For example for high performance withlarge objects, each layer may be striped separately, but for small filesor where performance is less critical, multiple layers may share astripe group. This method of separating layers into distinct stripegroups can also give the effect of increasing storage bandwidth, andeven in file systems without specific striping support, these methodscan give the effect of striping by storing each layer as a separatefile, with the layers being read in parallel.

Some other processing considerations and variations in the methodinclude:

-   -   Possible use of auxiliary resources, such as a separate        network-connected PC to assist in the required processing    -   Structuring the reorder processing (and data structures) such        that it is interruptible without loss of integrity, so that a        file that is partly reordered and partly not reordered can be        played without apparent discontinuity, and without undue        redundancy in storage usage.    -   Re-ordering might be done on a just-in-time basis, when space is        running out, to peel off one or more layers and save the rest.    -   If the input is not in scalable form, but is coded in a format        that is adaptable to scaling, such as various MPEG versions, the        decode/re-encode process may be subject to processing        simplification, since the basic transforms, quantization and        motion estimation encodings may be reused without duplicating        the original processing.

An additional consideration is that lossy compression algorithms areoften characterized by a quality metric parameter that correlates withperceived quality. This target quality parameter can preferably beapplied in the re-ordering or partitioning of enhancement layer datadescribed here. Essentially one can chose a partitioning that segmentsinto equal amounts of storage, or one that segments into equal losses ofquality. One can also use a composite method that considers bothfactors, such as, for example, one that creates a given layer only if itwill save an amount of storage that exceeds some threshold level ofefficiency relative to the quality loss. Coding of the presence orabsence of such layers, and of the associated efficiency metric for eachlayer, could be included in the file format, and could also beseparately tracked by the storage management system, to aid in thedecisions of a reclamation algorithm (which items, when, for how muchspace).

Of course it is desirable that content be provided in a coding formatthat provides scaleable layers, since the methods described aboveexploit that built-in scalability. Nevertheless, this method can also beapplied to content that is not obtained in layered form. In this caseadditional steps are needed to first transcode the data into a desirablelayered coding format. This adds processing overheads, but does so onlyonce, at the time the file is first written. These steps are preferablydone as the data is received, possibly while it is being processed forreal-time presentation, and before it is stored, and that enablesefficiencies in use of processing, work areas, buffers, and input/outputactivity that can be achieved using methods well known to those skilledin the art. Once converted, the object is saved in a layer-orderedformat suited to progressive deletion, and any number of progressivedeletion steps can be performed thereafter without need for any furthertranscoding steps. Such methods can be applied to analog sources aswell. This approach of layering-on-receipt may be very widely useful inthe near term, such as in DVRs for use with conventional analog ordigital broadcast, while use of scalable coding in source content is notyet widespread. Some current DVRs store analog TV as MPEG-2 by encodingit locally using coding schemes controlled within the DVR, and themethods described here can preferably take the form of softwareextensions to those existing coding and storage facilities. Similarly,such a local transcoding process could also be efficiently performedduring any presentation process.

Preferably data would be stored in a scalable re-ordered format at thetime of recording. Once the data is received (and any decoderconstraints are addressed) there is no need to maintain anon-partitioned, real-time broadcast order, and an object can be storedin the archival order suited to progressive deletion. Alternatively, dueto timing and resource issues, it can be desirable to do archivalreordering in a separate pass, some time after an initial capturerecording. An archiving system could manage this as a separate process,or in conjunction with replay, in which the original non-reorderedversion is read, reordered, and re-stored. This could be done as abackground task, based on resource availability and schedule parameters,such as overnight.

Full exploitation of these methods would benefit from provision to theuser of a comprehensive software-based archive manager service thatwould support any of a variety of media content types and of receptionand presentation systems. Such a service would preferably allowprogressive deletion to be done both manually and under automaticcontrol, and would preferably include a suite of management tools anduser agents. The service would preferably organize items into classes,set management policy rules (grouping them into rule families), trackusage patterns, and manage how the rules are applied to both the classesof items and to selected individual items. Such tools would preferablycombine artificial intelligence and learning with user controlled rulesand explicit commands. These methods would preferably adapt dynamicallyto changing levels of storage supply and demand. This facility wouldpreferably manage an entire hierarchy of personal storage, to managecaching for user subsystems and shared storage system servers, andmigrate items from high-speed direct-access storage to archival media(such as a DVD-R jukebox), as well as manage backup and retentionpolicies across all of a user's media assets. The methods of thisfacility would also preferably integrate or inter-work with other mediacontent archiving and management methods such as those used in existingDVRs and those described in U.S. Pat. Nos. 6,236,395, and 6,324,338.

Storage management algorithms that apply progressive deletion under richcontrols such as in such an archiving system might preferably beparameterized to specify rates of progressive deletion over time, andbreakpoints at which the rate changes or holds. Thus some items mightdegrade at one rate for a period, then another for a longer period.Other items might degrade only to a set level and then be held at thatlevel. Truly archival quality items would be set to not degrade at all.

There are parallels in the application of such policies to the use ofhierarchical storage, virtual storage, and cache management algorithmsfor determining which items to delete (completely) from an upper levelof hierarchy, such as least-recently used, lowest rated, least oftenlinked to, least used, etc. Similar deletion priority rules mightdetermine the decay rates to be applied to different items or classes ofitems. Key differences are that cache (hierarchical) management assumesthat the cache (higher level) is supplementary to a backing store (lowerlevel) that retains all items, so that deleted items can be restored (atsome performance cost), and that deletion from a cache is of an entireobject (or of storage unit without regard to its content associations),and not progressive with regard to the content of the data stored.Preferably, in an embodiment of progressive deletion applied inconjunction with a possible hierarchical storage system applied to mediaas proposed here, progressive deletion might be combined with such cachemanagement methods to offer additional flexibility in storage spacemanagement.

In broad terms, this approach can be though of as an extension of thecore idea of lossy compression, which is to discard the elements ofleast value to human perception. The new dimension here is time, asviewed across the lifetime of a collection of stored items. Theimportance of an item tends to decrease over time, and this approachexploits that principle, by applying loss selectively as items age (tothose items which are oldest, and most devalued by age). Progressivedeletion might preferably be done not only on explicit command tospecific objects, but by automated archive management processes thatapply rules for pruning back entire collections of objects. Standardprofiles might be used for classes of content (with the option ofspecific override), to specify both the rate(s) of decay and theendpoints/breakpoints, such as these purely illustrative examples:

-   -   Time-sensitive TV broadcasts, such as news: very rapid decay        over days, down to low quality    -   Routine TV broadcast series: rapid decay over weeks, down to        moderate quality, and slower thereafter    -   Favorite TV broadcast series: no decay for 1-2 weeks, followed        by moderate decay over weeks-months    -   Movies, grouped in 3-5 levels: moderate decay over months        (longer as large archives become practical), holding at a        moderate quality level    -   Home movies, grouped in levels: varying moderate decay over        months-years, holding at a range of qualities    -   Personal photo collections, grouped in levels: varying moderate        decay over months-years, holding at a range of qualities        Tools might be provided to facilitate periodic review and triage        of items, on a scheduled basis, and as storage becomes        constrained. These tools might include display of metadata        descriptors and preview aids, such as thumbnails or video clips.        An initial triage cycle (before and/or after recording) would be        most important to set item-specific parameters and category        selections.

Preferably the system would be structured to allow for creation of userprofiles that define key parameters relating to the above and able to dothat based on explicit entry or learning about user behavior, and wouldallow for balancing of the needs and preferences of multiple userssharing an archive and its storage resources, such as a family or officegroup. Some users may have higher priority, both as to retentionpolicies for items they prefer, and as to level of authority to setpolicies and to allocate and manage space (such as to limit children'suse). To the extent that user profile data is available in a standardformat, the system would preferably accommodate that format.

Preferably an archive management system would integrate broaderfacilities, such as controls over initial levels of compression, andbroad functions for the management of storage hierarchies, includingdisk, tape, DVD, and other storage media, as well as facilities forbackup, security, privacy, access control and sharing, update integritycontrol, editing and version control, etc.

-   -   Controls may be structured to associate policies for each of the        control mechanisms with the types of objects and with the        categories of objects (including individually identified        objects).    -   Preferably such policies for each control method would be        congruently specified and based on consistent groupings of items        by type and category, such that a single association of an        object into a category would be sufficient to ensure coherent        management policies for all control mechanisms applied to it        (such as initial compression level, use of progressive deletion,        migration to lower-level storage, etc.).    -   Such control policies may be defined by the provider of the        object, by the provider of the storage management system (or its        components), or by the user, or obtained from others.    -   Preferably a system of policies and rules would combine default        schemes established by system providers (or others) that may        work in conjunction with content provider suggestions, but        subject to systematic modification by the user. This would        enable the user to accept the offered defaults, to customize the        rules on a generic basis to apply to defined types and        categories, and to modify policies for specific objects or        groups of objects.    -   The system would preferably also enable compound object groups        consisting of associated objects to be managed together using        coherent policies, reflecting linked valuations, but applying        media-type-specific actions. This would include rich multimedia        such as enhanced TV with associated Web content or secondary        video streams, or collections such as a series of music pieces        grouped into a program (as a Disk Jockey compilation).    -   Preferably these rules and policies would be specified in a        standard format, such as Rule ML (Rules Markup Language), Simple        Rule Markup Language (SRML), Business Rules Markup Language        (BRML), Structured Rule Language (SRL), ILOG rules, or other        agent-oriented languages, and preferably one based on XML, and        preferably would be interoperable (or at least translatable) to        work with a wide range of storage systems (and all consumer        media object types).    -   Also, preferably, these rules and policies would be structured        to separate system-specific information and policies, such as        storage size and characteristics, so that system storage        facilities could be changed or expanded and the storage        management policies to be used would adapt automatically with        little or no need to rework or re-tune them.

The approach proposed here differs from HSMs as used in largeenterprise-class data centers which address analogous problems ofefficiently storing data for varied data processing applications withvarying usage and retention requirements. The methods applied in anenterprise system must be adapted as discussed herein to the differentmedia object types and categories, storage devices, and controlrequirements of a consumer home media system, and to the very simple yetpowerful user interface required by an individual consumer or family. Akey challenge of this adaptation is that such consumers or familymembers will act in the dual roles of both users and administrators ofsuch a system, and will need to be able to set and manage rules andpolicies for their stored objects for themselves. It should be notedthat some aspects of conventional enterprise storage management and HSMmay be applicable to managing the central source archives that supportserver-side distribution of multimedia content, such as for remote videoon demand services, but that is still an enterprise environment, andunlike a home (personal/family) environment, as addressed by theproposed system.

A key method for addressing differences from enterprise systems is acomposite policy definition process which applies pre-specified policydefaults that are defined by the system or content providers or thirdparty support services, and subject to modification by the user. Suchmodification can be made easy by specifying the default policies andcategories to the user in simple terms, and allowing the user to simplyaccept them, for a user to select among some simple variations, or foran advanced user to dig deeper into the details of the rules andoverride specific parameters and relationships. Such systems wouldinitially need to be very simple, but over time, a significantpopulation of users could gain sophistication and seek more advancedcontrols.

Adoption of a HSM system for consumer media centers on appropriatepolicies for:

-   -   when to migrate object sets to lower-level media which may be        larger and cheaper, but slower and not always mounted    -   when and how to access objects not immediately accessible        (including automatic and manual loading)    -   whether to reverse migrate objects to faster devices for more        effective use.        Here again, policies would be set based on the type and category        of the object, subject to user override.

Expanding further on the kinds of categories that might be useful in apersonal/family media archiving system, a category structure might bedefined in terms of multiple dimensions

-   -   families of object content categories, such as movies, TV        series, TV news, music, etc    -   source object quality categories, such as HDTV, DTV, VCR, VGA        (Video Graphics Adapter), ¼-VGA, SVGA (Super VGA), CD-audio, MP3        (lossy) audio, etc.    -   longevity categories, such as permanent, long, short    -   replaceability categories, such as impossible, difficult, easy    -   personal importance categories, such as maximum, high, medium,        low        The rule set might then define policies for each combination of        categories, and this would facilitate desirable behaviors, such        as high importance HDTV being managed differently from high        importance ¼-VGA. The user could then change the rules for        handling objects within a category combination, or change the        rules defining what objects fall into what categories. Such        changes could be continuing or for a specific time, and for all        objects or just specified objects (or categories).

Preferably a flexible facility might be provided to perform anynecessary coding/re-ordering functions, and such a facility could beinterfaced to other subsystems, such as its recording facility and itsarchive management facility, via a well-defined and flexibleApplications Programming Interface (API). Such a facility could be adistinct software product, usable by any media system, including DVRs,gateway servers, and Storage Area Networks, or by other devices, andcould be applied on each such device, or by using a shared serverfacility.

The essence of such a facility would be to:

-   -   input a stream, either as received from a remote source or from        a file system;    -   interpret it (decoding and recoding if needed);    -   optionally output a version of the stream for presentation;    -   re-order it;    -   store it; and    -   plus, either separately or during the above, to delete one or        more layers.

A wide variety of variations in operation can be viewed in terms ofcombinations of basic options and parameters that might preferably bespecified using the API.

-   -   Input: layered or not. Preferably any input stream format could        be used, with processing varied accordingly.    -   Archival output: layered or not, and what format options.        Preferably for progressive deletion, the output would be        layered, but in cases where real-time processing resources were        limiting, this could be deferred to a later non-real-time cycle.    -   Presentation: active or not. Depending on the usage mode, the        user may wish to view the content as it is being recorded (or        re-ordered) or not.    -   Layering: single EL or specified number or size or quality level        of EL layers to be re-ordered to and stored. This may usefully        vary depending on the specific data types, coding algorithms,        archive management methods, and user objectives.    -   Stripping/deletion: what number of layers is to be deleted. This        will depend on the amount of storage to be released.        Alternatively, deletion may be specified in terms of equivalent        or related metrics, including amount of space, fraction of        space, amount of quality, fraction of quality, and the like.        -   In cases where viewing is not done, if the stored file is            already formatted for progressive deletion, stripping may be            done by simple deletion.        -   In cases where the file is not yet formatted for progressive            deletion, a re-ordering pass would typically be required.        -   In cases where the content is being viewed, situational            rules may govern whether it is to be deleted entirely, by n            levels, or not at all.        -   In cases of background re-ordering, zero stripping might be            selected.            This can yield a highly flexible facility that can be used            to perform varying combinations of processing to manage an            archive under widely varying conditions and rule sets.

This new approach to management and compression for archival storage ofmedia objects provides compressed storage formats that can decay ordegrade gradually over time by releasing space, at some cost in quality,while still retaining the ability to view or present the object. Thusinstead of dropping one movie to record another, for example, a storagesystem might reduce the quality of five movies to make room for one newone. Similarly, if one fell behind in viewing installments of a serial,it might often be preferable to lose some fidelity than to lose entireepisodes. These methods further enable such reductions to be done ahighly efficient manner.

Although the invention has been described in detail in the foregoingembodiments, it is to be understood that the descriptions have beenprovided for purposes of illustration only and that other variationsboth in form and detail can be made thereupon by those skilled in theart without departing from the spirit and scope of the invention, whichis defined solely by the appended claims.

1-32. (canceled)
 33. A method for managing data storage capacity,comprising: storing a plurality of objects in machine readable formathaving at least a first layer of high-significance data and a secondlayer of separately-deletable low-significance data; determining acondition for deleting the second layer from each of the plurality ofobjects while retaining the first layer; and deleting the second layerautomatically upon satisfaction of the condition, whereby the objectsare still accessible by a user. 34-56. (canceled)
 57. The method ofclaim 33, further comprising: receiving the object from at least one of:a communications transmission, a storage device, a capture/creationdevice, and a second storage device on a computer network.
 58. Themethod of claim 33, wherein said deleting substantially preserves theperception of the object without apparent truncation.
 59. The method ofclaim 33, wherein said deleting is accomplished without re-storing thefirst layer.
 60. The method of claim 33, wherein said deleting isaccomplished by changing directory entries in a file management system.61. The method of claim 33, wherein said layers may correspond to atleast one of: spatial data, temporal data, and quality data. 62-72.(canceled)
 73. A method for transmitting data, comprising: generating anobject; and transmitting the object to a user in a format havingplurality of separately-deletable layers of data, with each layer beingof progressively decreasing significance.
 74. The method of claim 73,wherein at least two said layers are separately addressable for deletionfrom a lower significance to higher significance.
 75. The method ofclaim 74, wherein a lowest significance layer is deleted before a highersignificance layer.
 76. The method of claim 73, wherein the plurality oflayers correspond to quality levels of data such that deletion of thelower layer of data results in a minimal impact on a perception qualityof the object.
 77. The method of claim 73, wherein said plurality oflayers are stored as separately-addressable file structures associatedwith the object.
 78. The method of claim 73, wherein the object istransmitted to at least one of: a computer memory device, a personalvideo recorder memory, a music storage system, a video storage system, atelevision reception device and a home entertainment system.
 79. Themethod of claim 73, wherein the object is at least one of: a video filehaving at least one frame, an audio file, a multimedia file and astreaming multimedia file. 80-85. (canceled)
 86. A method of storing anobject in a storage facility that may lack available space sufficient tostore the object comprising: prior to an object storage request,determining a set of storage units used for currently stored objectsthat could be reallocated without degrading any of said objects beyondspecified thresholds; receiving an object storage request; storing theobject in the pre-determined storage units, in response to said request.87. The method of claim 86, wherein said storing uses no more than thenumber of physical input/output operations that would have been requiredto store the object if the storage units had been unallocated.
 88. Themethod of claim 86, wherein said set of storage units remainspotentially accessible for retrieval of said currently stored objectsand wherein said determination may be changed until said storage requestis received. 89-92. (canceled)